Distributed representations, simple recurrent networks, and grammatical structure

Distributed Representations, Simple Recurrent Networks, and Grammatical Structure

Tail-Recursive Distributed Representations and Simple Recurrent Networks

Representation poses important challenges to connectionism. The ability to structurally compose representations is critical in achieving the capability considered necessary for cognition. We are investigating distributed patterns that represent structure as part of a larger effort to develop a natural language processor. Recursive Auto-Associative Memory (RAAM) representations show unusual prom...

Learning Subsequential Structure in Simple Recurrent Networks

We explore a network architecture introduced by Elman (1988) for predicting successive elements of a sequence. The network uses the pattern of activation over a set of hidden units from time-step t-l, together with element t, to predict element t+ 1. When the network is trained with strings from a particular finite-state grammar, it can learn to be a perfect finite-state recognizer for the gram...

Boundary effects in the linguistic representations of simple recurrent networks

representation of surface form is erased from immediate memory" (Garnham, 1985, p. 204). The goal of the study described here is to discover if these phenomena can be accounted for within a connectionist framework, and without invoking an external memory management system. This paper describes a number of simulations which show that SRN representations exhibit interactions between memory and se...

Generalization, simple recurrent networks, and the emergence of structure

The problem is that most human behaviors seem to lie somewhere between these two possibilities. Neither lists nor rules capture the richness of cognitive behaviors, which are populated by what Plaut, McClelland, Seidenberg, and Patterson have called “quasi-regular” domains. There are underlying generalities to most of what we do (or think), but the generalities are typically partial and tempere...